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ABSTRACT 

To support teachers in enhancing ocean literacy, the Center for Microbial Oceanography: Research and Education (C-MORE) 
has developed a series of portable, hands-on science kits on selected topics in oceanography. This paper provides an overview 
of kit content, describes how the kits were developed, and evaluates their efficacy as a curriculum supplement in formal 
classroom environments by using two studies, a qualitative assessment of teacher experience and a quantitative assessment of 
student learning. The teacher surveys (n = 45) indicate the kits were used in a wide range of class types and grade levels, 
describe the kits as useful classroom tools that actively engaged students and resulted in meaningful learning, and strongly 
indicate that the teachers plan to continue to use the kits in the future. The student learning assessment (n = 1,236) employed 
a pretest, posttest 1, and posttest 2 methodology. The pretest and posttest 1 were given immediately before and after 
instruction, respectively; posttest 2 was given at least 2 weeks after instruction. Engaging with the science kits resulted in 
significant knowledge acquisition (pretest-posttest 1 mean differences of 0.21-0.41) and retention (pretest-posttest 2 mean 
differences of 0.18 to 0.39), with significance defined at the a = 0.05 level. In the 2 weeks after kit instruction, all kits showed a 
slight, nonsignificant loss in knowledge (mean differences of —0.01 to —0.04). Together, the teacher and student evaluations 
indicate that the C-MORE kits are effective classroom tools that can serve as a model for hands-on curriculum supplements. 
© 2013 National Association of Geoscience Teachers. [DOI: 10.5408/12-336.1] 
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INTRODUCTION 

Ocean and Science, Technology, Engineering, and 
Mathematics Literacy 

Virtually every aspect of modern life is influenced by 
science, technology, engineering and mathematics (STEM). 
Engaged citizens need to possess at least basic STEM literacy 
to make informed decisions about everything from personal 
health to public policy. Alarmingly, a third of U.S. middle 
school students failed to meet the basic achievement level in 
the most recent national science assessment (National 
Center for Education Statistics, 2012). These and similar 
reports highlight the need for vastly improved STEM 
education (e.g., National Research Council, 2010, 2011, 
2012; National Academy of Engineering and National 
Research Council, 2012). 

A key component of science literacy is ocean literacy. 
The ocean shapes Earth's features, makes our planet 
habitable, plays a pivotal role in weather and climate, 
supports great diversity of life, and can drive the creation of 
new technologies. The oceans are inextricably linked to the 
Earth, atmosphere, and life, and a basic knowledge of ocean 
science is critical to understanding global processes (Steel et 
al., 2005; National Geographic Society and National Oceanic 
and Atmospheric Administration, 2006; Strang et al., 2007). 
Unfortunately, marine science topics are not yet widely 
integrated into state and national standards, education 
frameworks, or teacher training programs. Therefore, while 
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ocean sciences could easily be integrated into a range of 
science classes, they generally are not (Walker et al., 2000; 
Lambert, 2006; Clay et al., 2008; Schoedinger et al., 2010; 
Tran et al., 2010). 

The development of the Next Generation Science 
Standards is underway to guide the future of science 
education (Achieve, Inc., 2012). The Framework for K-12 
Science Education (Framework) has recommended that the 
Next Generation Science Standards be built around three 
major dimensions: (1) scientific and engineering practices, 
(2) crosscutting concepts that unify the study of science 
through common applications across fields, and (3) core 
ideas in the four disciplinary areas of physical sciences, life 
sciences. Earth and space sciences, and engineering- 
technology-applications (National Research Council, 2012). 
Ocean processes are among the crosscutting concepts cited 
in the Framework, and their inclusion is guided by the 
Ocean Literacy Principles (National Geographic Society and 
National Oceanic and Atmospheric Administration, 2006). 
Thus, the demand for flexible ocean science curricular 
materials that can be incorporated into a range of course 
types is likely to increase in the near future. 

Science Kits and Hands-On Learning 

To effectively learn science, students need to learn more 
than just facts and theories. Rather, they need to construct 
their understanding through an iterative, collaborative 
process that builds and reshapes prior knowledge (National 
Research Council, 2000, 2005, 2012). The most effective 
lessons tend to be hands-on and inquiry-based (Shymansky 
et al., 1990; Olson and Loucks-Horsley, 2000; Young and 
Lee, 2005), place-based (Gruenewald, 2003; Sobel, 2004), 
and/or relevant to current issues (Ballantyne et al., 2001). 
Students who participate in hands-on activities focused on 
scientific inquiry score more highly on national science tests 
(National Center for Education Statistics, 2012). 
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Unfortunately, hands-on science education tends to be 
resource intensive, and many schools are experiencing 
financial shortfalls. One way that teachers can keep up to 
date with equipment and supplies is with science kits. 
Science kits contain all (or nearly all) of the equipment, 
supplies, and curricular materials needed to investigate a 
particular science topic. The most well known science kits 
are commercial—for example, Full Option Science System 
(Delta Education), Science and Technology for Children 
(Carolina Biological Supply Company), and Insights (by 
Kendall Hunt) (Lopez and Schultz, 2001). These kits, which 
can include 6 or 8 weeks of curriculum, have been shown to 
be effective at increasing student achievement and promot¬ 
ing positive attitudes toward science (Dickerson et al., 1996; 
Stohr-Hunt, 1996; Houston et al., 2003; Young and Lee, 
2005). Unfortunately, they exceed the budget of many 
classrooms. 

In recognition of the efficacy of commercial science kits, 
scientific and educational organizations are starting to 
develop their own kits as part of their outreach mission 
(e.g., Aquarium of the Pacific, 2012; Cascades East, 2012; 
Center for Microbial Oceanography: Research and Educa¬ 
tion, 2012; Missouri Botanical Garden, 2012; Ohio State 
University Extension, 2012). Unlike commercial kits, which 
are typically designed as stand-alone curriculum, these kits 
generally contain hands-on lessons and activities to supple¬ 
ment a teacher's existing curriculum. Little research, 
however, appears to have been done as to whether these 
"curriculum supplement" kits are effective educational tools. 

This paper focuses on a collection of "curriculum 
supplement" kits created by the Center for Microbial 
Oceanography: Research and Education (C-MORE) on a 
range of topics in ocean science. First, we provide an 
overview of kit content and describe how the kits were 
developed, revised, and distributed through a system of 
lending libraries. We offer this information as a model for 
teachers or organizations interested in developing their own 
kits. We then evaluate the efficacy of the science kits as 
curriculum supplements in formal classroom environments 
through two studies, an assessment of teacher experience 
and an assessment of student learning. Our research 
questions are (1) What were the teachers' experiences in 
engaging with the kits? and (2) Does engagement with the 
science kits increase student knowledge and build ocean 
literacy? 

C-MORE SCIENCE KITS 

In 2009, C-MORE released a set of hands-on science kits 
on various topics in oceanography (Center for Microbial 
Oceanography: Research and Education, 2012). These "labs 
in a box" are designed to be curriculum supplements for a 
range of science courses and grade levels. Although the 
lessons in each kit make up a unit, many of the activities are 
discrete and can be rearranged to fit various curricular 
objectives and time constraints. No prior content knowledge 
or specialized training is required to use these kits, but 
teachers with limited background in ocean science are 
advised to review all resource materials provided prior to 
instruction. 


Each kit contains hands-on supplies and paper and 
electronic materials, which include standards-based lesson 
plans, narrated and non-narrated versions of Microsoft 
PowerPoint presentations, resource materials (such as 
videos, data sets, and readings), teacher evaluation surveys, 
and supply checklists. All paper and electronic materials are 
available online (Center for Microbial Oceanography: 
Research and Education, 2012) and may be freely repro¬ 
duced for educational, noncommercial purposes. To main¬ 
tain the integrity of the assessment, answer keys to 
assessment materials are not provided online but may be 
obtained by contacting the C-MORE science kit manager 
(kits@soest.hawaii.edu). 

Fourteen science kit lending libraries were established in 
the four states where C-MORE education efforts are 
concentrated: Hawaii, California, Massachusetts, and Ore¬ 
gon. Teachers (including informal science educators and 
homeschool parents) can borrow kits at no cost. First, they 
reserve the kit from a specified lending library location by 
using an online reservation system. Then, they are contacted 
by the lending librarian to arrange a pick-up time. As of 
December 2012, the kits have been used with at least 30,000 
students and 325 unique educators from 235 organizations at 
grade levels ranging from kindergarten to college and at 
teacher development workshops. 

Teachers outside of lending library service districts can 
access paper and electronic materials online to teach lessons 
by using supplies already on hand or that do not require 
hands-on materials. At least 58 teachers from 25 states and 
five foreign countries have done this, as evidenced by their 
request for answer keys to assessment materials. Alternately, 
teachers can purchase supplies, print the paper materials, 
and physically replicate part or all of a science kit. We are 
aware of 26 teachers from eight states who have done this. 
Teachers interested in replicating kits are welcome to contact 
the C-MORE kit manager for advice and assistance. 

All C-MORE science kits are aligned with the state 
education standards and benchmarks for Hawaii (Hawaii 
Standards Database, 2005), California (California Depart¬ 
ment of Education, 2003), Massachusetts (Massachusetts 
Department of Education, 2006), and Oregon (Oregon 
Department of Education, 2009), as well as with the Ocean 
Literacy Principles (National Geographic Society and Na¬ 
tional Oceanic and Atmospheric Administration, 2006). 
Hawaii is the only one of these states with specific ocean- 
related standards for marine science courses. 

Kit Content Descriptions 

The five kits examined in this study are summarized 
below. Detailed lesson plans for each kit are provided as 
supplemental online documents. 

(1) Ocean Acidification. The primary objective of this 
two-lesson kit is to introduce students to ocean 
acidification, the process by which the ocean is 
becoming increasingly acidic (Bruno et al., 2011). In 
brief, ocean acidification is caused by increasing 
carbon dioxide (CO 2 ) levels in the atmosphere. 
Some of this CO 2 dissolves in the ocean, where it 
combines with water to form carbonic acid, a weak 
acid. A more acidic ocean poses threats to marine 
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organisms. Lesson 1 (60 min) includes a simple, 
hands-on experiment demonstrating the impact of a 
weak acid on carbonate sand, a short presentation, 
and optional readings with worksheets. In lesson 2 
(60 min), students conduct an experiment with 
Vernier probes to simulate the process of ocean 
acidification. They activate yeast, measure the CO 2 
produced by yeast respiration, bubble CO 2 through 
water, and measure the resultant pH decrease that 
occurs as carbonic acid forms. After completing this 
kit, students should know about ocean acidification, 
the basic chemical relationship between CO 2 and 
water, the pH scale, and potential impacts of ocean 
acidification. (See Ocean Acidification Kit lessons, 
available online at http://dx.doi.org/10.5408/12- 
336sl.) 

(2) Ocean Conveyor Belt. The objective of this four- 
lesson kit is to introduce students to fundamental 
oceanographic concepts such as water density, 
ocean circulation, nutrient cycling, and variations 
in the chemical, biological, and physical properties 
of seawater through hands-on and computer-based 
activities. In lesson 1 (70 min), students learn how 
water density changes with temperature and salinity 
by creating a stratified water tank. In lesson 2 (70 
min), students learn how changes in water density 
drive global ocean currents by mapping and making 
a paper model of thermohaline circulation. Lesson 3 
(70 min) introduces students to long-term oceano¬ 
graphic time-series data (Hawaii Ocean Time- 
series, 2013) to demonstrate how physical, chemical, 
and biological variables change with depth and 
time. Students graph and interpret seasonal changes 
in ocean temperafure, phytoplankton, and nutrient 
abundances. Lesson 4 (70 min) integrates every 
principle in the kit about ocean circulation, biogeo¬ 
chemistry, and data interpretation. Students use 
online oceanographic databases to graph, analyze, 
and interpret data about nutrient concentrations in 
the Atlantic and Pacific Oceans. After completing 
lessons 1 and 2, students should know how 
temperature and salinity affect water density, that 
most of the ocean is stratified, that the ocean has a 
global circulation pattern called the ocean conveyor 
belt, and that ocean circulation influences climate. 
After completing lessons 3 and 4, students should 
know the relative age of ocean water, general 
nutrient relationships in the open ocean, and the 
basics about nutrient cycles. (See Ocean Conveyor 
Belt Kit lessons, available online at http://dx.doi.org/ 
10.5408/12-336s2.) 

(3) Plankton. The objective of this four-lesson kit is to 
introduce plankton and the crucial role they play in 
the marine food web. Most students are surprised to 
learn that there are microscopic organisms all 
around them when they swim in the ocean. In 
lesson 1 (40 min), students learn about phytoplank¬ 
ton ("plant-like" plankton) and their adaptations for 
survival. In lesson 2 (45 min), students design their 
own phytoplankton, based on observation and 
imagination. In lesson 3 (50 min), students collect 


a live zooplankton ("animal-like" plankton) sample 
with a net and investigate it with the digital 
microscope provided. In lesson 4 (60 min), students 
learn about environmental factors that affect phy¬ 
toplankton growth and distribution, and run a 
computer simulation to model phytoplankton 
blooms. After completing lessons 1 through 3, 
students should have learned about phytoplankton 
and zooplankton, including their survival adapta¬ 
tions, their roles in the marine food web, and their 
influence on climate. After completing lesson 4, 
students should know about environmental factors 
that affect plankton growth. (See Plankton Kit 
lessons, available online at http://dx.doi.org/10. 
5408/12-336s3.) 

(4) Random Sampling. There are millions of microbes in 
a single drop of sea water, but how do scientists 
count them? The objective of this kit is to introduce 
random sampling, a key concept used to study the 
natural environment (Bruno et al., 2010). In lesson 1 
(50 min), students study the abundance and 
diversity of marine microbes. Differently colored 
beads represent different species of microbes, and 
the bag containing the beads represents the ocean. 
Working in groups, students randomly select a 
subset of beads from the "ocean" and compare the 
composition of their samples with that of the entire 
population. They learn that one does not need to 
count every individual to get a good representation 
of a population, provided that tbe sampling method 
is random. In lesson 2 (50 min), students are 
introduced to Microsoft Excel by inputting and 
graphing their lesson 1 data. In lesson 3 (50 min), 
students use chi-square statistics to assess how well 
their samples represent the total population. After 
completing lesson 1, students should know what a 
random sample is and how to randomly sample a 
population. After completing lesson 2, students 
should know how to make a simple Excel graph. 
After completing lesson 3, students should know 
how to use statistics to compare random samples to 
the population. (See Random Sampling Kit lessons, 
available online at http://dx.doi.org/10.5408/12- 
336s4.) 

(5) Marine Debris. This three-lesson kit explores marine 
debris and its environmental impacts. In lesson 1 (60 
min), students complete six hands-on activities to 
explore how marine debris can harm marine 
organisms, such as through entanglement and 
ingestion. In lesson 2 (80-90 min), students learn 
how surface ocean currents form and how currents 
can transport debris far from the source. They trace 
potential marine debris tracks by using the Ocean 
Surface Current Simulator, an online ocean surface 
current model (National Oceanic and Atmospheric 
Administration, 2013) to determine the geographic 
origin of the debris. In lesson 3 (60 min), students 
learn how some scientists study marine debris and 
engage in a discussion about what they can do to 
help with this important problem. After completing 
this kit, students would have learned the definition 
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of marine debris, its environmental impacts, that 
surface currents can transport debris across oceans, 
how surface currents form, and how they can help 
mitigate marine debris. (See Marine Debris Kit 
lessons, available online at http://dx.doi.org/10. 
5408/12-336s5.) 

Kit Development 

Each kit was developed through an iterative, collabora¬ 
tive process involving pilot tests and teacher feedback, 
during which the lessons were continually revised, as 
recommended by Briggs et al. (1991). First, researchers and 
educators brainstormed ideas and topics related to microbial 
oceanography of potential interest in the classroom. Then, 
we outlined possible ideas for standards-based lessons, 
incorporating technology and hands-on activities as much as 
possible. Much of the development stage involved research¬ 
ing and testing products to be used in the activities. Once we 
selected the best materials and experimental procedures, we 
wrote lesson plans to guide students—and teachers— 
through the experiments. All lessons and worksheets were 
field tested with a variety of student audiences in both 
informal and formal settings. 

Throughout kit development, we solicited feedback from 
classroom teachers, both informally on a one-on-one basis 
as well as through organized teacher focus groups, where 
teachers reviewed and commented on draft materials. 
Although they held a range of opinions, several "big- 
picture" patterns emerged. We learned that teachers were 
looking for versatile curriculum supplements, not replace¬ 
ment curricula. They wanted hands-on lessons that related 
to "hot" topics that could be incorporated into their existing 
curriculum and aligned with state standards. They were 
virtually unanimous in not wanting a prescribed curriculum 
targeted at a specific grade or subject. They were especially 
interested in connecting with university-based research 
endeavors and accessing "real" data sets. When reviewing 
specific kits, teachers were especially helpful in gauging 
whether the lessons would be of interest to students and 
identifying conceptual leaps that needed bridging. 

In response, we created science kits to explore "big 
ideas" that could be connected to a variety of science 
subjects and aligned them with state standards at a range of 
grade levels. For example, a fourth-grade teacher could use 
the plankton kit to ask students to "Explain how simple food 
chains and food webs can be traced back to plants," whereas 
a high school biology teacher might use the kit to discuss the 
important role of plankton to "Describe biogeochemical 
cycles within ecosystems" (Hawaii Standards Database, 
2005). 

Kit Production, Distribution, and Revision 

Once the content materials for each topic area were 
finalized, we gathered the supplies and physically assembled 
14 kits per topic area (one per lending library). To provide a 
sense of the scale of this effort, most of the kits contain 
supplies for five to seven student groups, so we purchased 
and/or created 70-98 individual items per group activity. 
Since most retailers do not stock large volumes of items, we 
had to scout out multiple supply sources and place 
numerous orders to accumulate sufficient supplies. Back¬ 


ordered supplies often held up kit assembly for weeks to 
months. Once all the kits were assembled, they were 
delivered or mailed to kit libraries. Lending librarians 
(science educators who volunteer to host the kits and 
support local teachers in kit usage) then had to be trained on 
kit use, either online, over the phone, or in person. The time 
to produce and release one set of 14 replicate kits took 
approximately 8-14 months. On the other hand, a teacher 
wishing to replicate a single kit could probably do so in a 
couple of weeks by using the supply lists in the kits or 
contacting the kit manager for assistance. 

In the above paragraph, the word finalize is used loosely. 
Once the kits were released and teachers began to borrow 
the kits, we began to receive teacher feedback. Some 
teachers completed and returned the teacher survey included 
in each kit. Others contacted the kit manager or other C- 
MORE members with ideas, suggested edits, and requests 
for additional content. After at least a year of classroom use, 
the Plankton, Random Sampling, Ocean Acidification and 
Marine Debris kits were recalled, updated, and re-released. 
Updates included adding new student worksheets, revising 
student instructions for clarity, updating presentations with 
new information, and including pre- and post-surveys that 
were not included in the original release. For the Random 
Sampling kit, new material was added to better connect the 
activity to marine microbes. For the Marine Debris kit (in 
which problems were reported using the online ocean 
surface current model), new instructions were provided for 
an upgraded version of the computer model (National 
Oceanic and Atmospheric Administration, 2013). A recent 
addition to the kit collection, the Ocean Conveyor Belt, was 
released in 2010 and has yet to undergo major revision. 
Once the Next Generation Science Standards are adopted, 
all kits will be aligned to these new standards. 

ASSESSMENT OF TEACHER EXPERIENCE 

As of December 2012, a minimum of 325 unique 
educators from 235 organizations borrowed at least one 
science kit, reaching at least 30,000 students. Their 
experience with the science kits was assessed though an 
optional teacher evaluation included with each kit. The 
survey had two parts. First, teachers were asked to state their 
level of agreement with seven statements by using a Likert 
scale. Then, teachers were asked to respond to six open- 
ended questions and provide general comments. Teachers 
could either complete a pen-and-paper survey included with 
the kit or an online version (http://www.surveymonkey.com/ 
s/CMORE-kit-evaluation). Hard copies of teachers' evalua¬ 
tions were input online for data compilation and analysis. 

Forty-five teachers, or 14% of the 325 teachers who 
independently borrowed the kits from a C-MORE lending 
library, completed the evaluation. Although low return rates 
are fairly typical with optional evaluations, we note that they 
can skew results, either toward high or low satisfaction. The 
respondents were from Hawaii (33), California (6), Massa¬ 
chusetts (3), and Oregon (1); two respondents did not 
indicate a location. Of the 30,000 students using the kit, 
approximately 16% were in elementary school (K-5th 
grade), 33% in middle school (6-8th grade), 40% in high 
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TABLE I: Teacher feedback on using the science kits. 


Question 

Mean ± SD (n = 45) 1 

Online kit reservation was easy. 

4.53 ± 0.80 

Picking up this science kit was difficult.* 

1.55 ± 1.08 

This science kit was easy to use. 

4.38 ± 0.87 

The teacher guide was difficult to follow.* 

1.49 ± 0.69 

My students enjoyed using this science kit. 

4.51 ± 0.75 

I would not borrow this science kit in the future.* 

1.48 ± 1.05 

I will borrow other C-MORE science kits. 

4.60 ± 0.80 


'Teachers indicated agreement with statements by using the following Likert scale: 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly 
agree. 

* = reversely worded questions; thus, low values indicate positive results. 


school (9-12th grade), and 5% in college. The remaining 6% 
of kit usage occurred at teacher professional development 
workshops and adult education programs. Apart from grade 
level, no demographic information was collected on students 
of these teachers who independently borrowed the kits. 
However, detailed demographic information was recorded 
for the 1,236 students who participated in the student 
evaluation study described below (see "Assessment of 
Student Learning"), including ethnicity, grade level, aca¬ 
demic level within their grade, and eligibility for free or 
reduced lunch. 

Likert Scale Assessment 

Teachers were asked to state their level of agreement 
with seven statements by using a Likert scale of 1 (strongly 
disagree) to 5 (strongly agree). Four statements were 
positively worded, so higher numbers correspond to more 
positive results. Three statements were reversely worded, as 
recommended by Nunnally (1978), so lower numbers 
correspond to more positive results. Teacher responses to 
Likert scale questions are detailed in Table I and summarized 
here; asterisks denote items that are reversely worded. In 
short, teachers found the kits easy to reserve (4.53), pick up 
(1.55*), and use (4.38). They found the teacher guides to be 
easy to follow (1.49*), and reported that their students 
enjoyed using the science kits (4.51). They indicated they 
would both borrow the given science kit again (1.48*) and 
would also borrow other science kits (4.60). These quanti¬ 
tative results indicate that the teachers who independently 
borrowed the kits had strongly positive experiences. 

Open-Ended Questions 

In addition to the Likert scale questions described 
above, teachers were also asked to respond to six open- 
ended questions to ascertain how they used the kits and to 
solicit their feedback and suggestions. 

(1) Are the time estimates given for each lesson reasonable? 
If not, please explain. Approximately two-thirds (25) 
of the 38 teachers who responded to this question 
indicated that the timing provided in the lesson 
plans was reasonable; the remaining third (13) felt 
the time estimates were too short. Some teachers 
reported that they modified lessons to fit their time 
constraints, and several noted that the time required 


to complete the lessons varied with class type. For 
example, "In lab, I gave directions on a PowerPoint 
and had students follow along. This was much faster 
[Ocean Acidification]," and "My class is a SP[ecial]- 
ED[ucation] class with students with high-function¬ 
ing autism, so I had to modify each lesson. I adapted 
each lesson (for SPED students), but [the timing] 
would work well in a general-ed[ucation] class 
[Plankton]." 

(2) How did you use this science kit? The responses from 
41 educators collectively indicated that the kits were 
being used in a variety of classes including biology, 
chemistry, Earth and space science, language arts, 
life science, marine biology, marine conservation, 
physical science, and statistics. In these classes, the 
kits were used "as a lead-in to sampling methods 
for rocky intertidal monitoring;" "to discuss chem¬ 
ical reactions, chemical properties, conservation of 
matter and the impact of humans on the environ¬ 
ment," "to introduce classification, reproduction, 
currents, heredity, cycles of matter and energy, 
interdependence, inquiry, and unity and diversity," 
"to expand on the topic of scientific investigation," 
"to introduce ocean food webs, plankton, taxonomy 
and as an introduction to the microscope," "to 
introduce climate change," "to give students ideas 
of possible research projects they could develop," 
"to help students understand oceans' currents," "to 
reinforce simple random sampling and introduce 
some Excel features," "as part of a unit on ocean 
chemistry before ocean cycles," and "as an intro¬ 
duction to ocean circulation and geological ocean¬ 
ography." 

(3) Did you use the entire science kit? If you omitted any 
lessons or activities, please list which ones and explain 
why you skipped them. Forty teachers responded to 
this question. Fewer than half (18, or 45%) used the 
entire kit. Seven respondents (18%) skipped one 
lesson, 5 (13%) skipped two or more lessons, and 10 
(25%) modified the lessons. Reasons indicated for 
not using the entire kit included time constraints 
(e.g., "I ran into scheduling issues, otherwise I 
would have used the entire kit [Ocean Conveyor 
Belt]"), content considerations (e.g., "I skipped 
lesson 3 because I'm not introducing those topics 
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TABLE II: Numbers of students participating in the science kits evaluation. 


Science Kit 

Participating n 1 

Listwise Valid n 2 

ELL n (%) 3 

IEP n (%) 4 

Ocean Acidification 

187 

155 

38 (20%) 

18 (10%) 

Ocean Conveyor Belt 

327 

270 

10 (3%) 

11 (3%) 

Plankton 

228 

196 

4 (2%) 

11 (5%) 

Random Sampling 

220 

201 

4 (2%) 

11 (5%) 

Marine Debris 

274 

246 

21 (8%) 

12 (4%) 

Total 

1,236 

1,068 

77 (6%) 

63 (6%) 


Participating n = number of students who participated in at least one survey assessment for a given science kit. 

2 Listwise valid n = number of students who completed all three survey assessments (one pretest and two posttests) for a given science kit. 
3 ELL = English language learners. ELL data were provided by teachers and not linked to specific survey data. 

4 IEP = Individualized Education Program students. IEP data were provided by teachers and not linked to specific survey data. 


until second semester [Random Sampling]," and "I 
was using it as a survey and introduction and I 
didn't require the students to do everything in the 
kit [Marine Debris]"), and grade level ("I skipped 
[lessons] 3 and 4 due to age level [Ocean Conveyor 
Belt]"). 

(4) Were your students involved and interested in the 
science kit activities? All 45 respondents to this 
question replied positively. Sample comments in¬ 
clude: "Yes, they even took pictures of their product 
during and at the end of the assignment [Ocean 
Conveyor Belt]", "Definitely, they were fully en¬ 
gaged. It is evident by their thoughtful questions 
and the attention they paid to completing the data 
sheet [Plankton]", and "Yes, they were eager to see 
if their predictions were accepted or rejected [Ocean 
Acidification]." One teacher responded, "I decided 
to scrap my regular introduction to the microscope 
lab and instead combined it with the zooplankton 
lesson [lesson 3] in which the students looked at our 
bay water ... The water was filled with phytoplank¬ 
ton and zooplankton. To quote several students and 
give you the general idea of how it went, 'This was 
the best science class I've ever had!' rang through¬ 
out the room. The kids were so excited—I was being 
pulled in 11 directions at once to go see what they'd 
found swimming in the bay water with their 
microscopes as they were individually calling out, 
'Mrs. —■, come see this!' [Plankton]." 

(5) Please suggest two things that could be improved. 
Thirty-seven respondents suggested both technical 
improvements and additional components that they 
would like to see included with specific kits. 
Suggestions for technical improvements included 
"New boluses for dissection, as the ones in the kit 
have been dissected quite a bit. Also new/more 
samples of marine debris which organisms have 
attached/made homes on [Marine Debris]," and "I 
would increase the copies of all the laminated work 
to at least 10-12 of each. With class sizes increasing 
and individual educational plans for diverse popu¬ 
lations, it's often better to have students work in the 
smallest groups possible for some activities [Plank¬ 
ton]," and "Being new to tech science equipment 
[Vernier], my students would have been able to 


accomplish the CO 2 and pH experiments more 
efficiently if the two experiments had come in 
separate Baggies [Ocean Acidification]." Sugges¬ 
tions for additional components included "I would 
have liked to have some actual prepared slides of 
some of the plankton [Plankton]," and "A bucket 
would have been useful to get seawater for diluting 
the sample [Plankton]." 

(6) Any other comments? Responses from the 26 teachers 
answering this question included "Marine Debris is 
an easy kit to connect with my students' daily lives," 
"The Ocean Acidification kit was great, my kids 
loved doing a real scientist science lab," and "The 
students were amazed at the world of plankton. 
Some of them have very little science background, 
and the lessons were designed so that they got as 
much out of them as the students who have more 
experience in science [Plankton]." 

These qualitative responses, together with the quantita¬ 
tive Likert scale data (shown in Table I and summarized 
above), strongly indicate high satisfaction with the science 
kits among teachers who independently borrowed the kits 
and taught the lessons. Naturally, student learning is the 
ultimate arbiter of high-quality instructional materials, and 
this is assessed in the next section. 


ASSESSMENT OF STUDENT LEARNING 

From August 2010 to February 2011, we conducted an 
assessment to determine whether any significant gains in 
student learning occurred as a result of engagement with the 
science kits. The assessment was conducted in partnership 
with two external evaluators and eight classroom science 
teachers at six middle and high schools in Hawaii. The eight 
teacher participants responded to a request posted on the 
Hawaii Science Teachers Association email list. The students 
were in grades 7-12 and spanned a range of academic levels, 
including general lower, mixed, honors, and gifted and 
talented. A total of 1,236 students participated in the 
evaluation, with the number of students participating in 
each individual kit evaluation ranging from 187 to 327. 

Table II summarizes the number of students who took 
part in each kit evaluation, including those who were 
English-language learners and Individualized Education 
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TABLE III: Knowledge acquisition: descriptive statistics and paired Student's t-tests of student scores on the pretest and posttest 1. 


Science Kit 

Pairwise Valid n 

Pretest 1,2 

Posttest l 1,2 

Difference 3 

Student's f-test 

p Value 

Ocean Acidification 

167 

0.48 ± 0.21 

0.81 ± 0.18 

0.33 ± 0.24 

17.71 

<0.001 

Ocean Conveyor Belt 

296 

0.42 ± 0.22 

0.64 ± 0.20 

0.21 ± 0.24 

15.24 

<0.001 

Plankton 

207 

0.45 ± 0.17 

0.85 ± 0.16 

0.40 ± 0.21 

27.38 

<0.001 

Random Sampling 

211 

0.55 ± 0.23 

0.77 ± 0.23 

0.22 ± 0.25 

12.77 

<0.001 

Marine Debris 

253 

0.60 ± 0.23 

0.87 ± 0.18 

0.27 ± 0.23 

19.01 

<0.001 


1- The pretest and posttest 1 were given to students immediately before and after science kit instruction, respectively. 
2 Values are means ± SD. 

difference = posttest 1 minus pretest. Values are means ± SD. 


Program students. The demographics of the participating 
students represent the diverse population of Hawaii, with 
31% of the students Native Hawaiian or Pacific Islander, 
29% Filipino, 20% Caucasian, 14% Asian, 3% African 
American, and 3% Hispanic. Almost half of the students 
(47%) participate in a free or reduced-price lunch program. 
Student ethnicity and economic status were determined 
from school demographic data (Accountability Resource 
Center Hawaii, 2012) and not tracked for comparisons 
between groups in this study. 

This study was reviewed by the University of Hawaii's 
Committee on Human Studies and determined to be exempt 
from Department of Health and Human Services Regula¬ 
tions, 45 CFR, Part 46. Nevertheless, parents were notified 
by way of a letter sent home with the students that their 
child(ren)'s class would be participating in an educational 
study. Parents submitted a signed consent form granting 
permission for their child(ren) to participate in the study. 
Additionally, students signed consent forms acknowledging 
their participation. Students whose parents declined consent 
or who declined to participate in the study themselves still 
participated in all science kit activities and surveys, but their 
data were not included in the results. 

Student knowledge was assessed by using part or all of 
the surveys included with each of the science kits. For three 
kits, our evaluation utilized the entire survey: Ocean 
Acidification (8 questions). Marine Debris (5 questions), 
and Random Sampling (5 questions). For two kits, our 
evaluation utilized only part of the survey, because these 
surveys are divided according to age group-appropriate 
lessons. As our goal was to assess the kits across a range of 
grade levels, we omitted upper-level surveys from our 
evaluation. For the Ocean Conveyor Belt kit, we only 
administered survey questions on lessons 1 and 2 (7 
questions). For the Plankton kit, we only administered 
survey questions on lessons 1-3 (10 questions). The surveys 
contain a combination of multiple choice and true-false 
questions intended to assess the knowledge, comprehen¬ 
sion, and application of topics presented in each kit. The 
survey used with the Plankton kit is included as an example 
(Appendix 1). 

The surveys were administered at three stages: (1) 
immediately prior to science kit instruction (pretest), (2) 
immediately after science kit instruction (posttest 1), and (3) 
approximately 2 weeks after science kit instruction (posttest 
2). At each stage, the identical survey was administered, and 
each completed survey was assigned a decimal grade 


corresponding to the percentage of questions answered 
correctly. A comparison of pretest and posttest 1 scores 
allows us to assess any knowledge gain that may have been 
acquired as a direct result of science kit instruction. A 
comparison of pretest and posttest 2 scores enables us to 
measure whether knowledge was retained 2 weeks after 
science kit instruction. A comparison of posttest 1 and 
posttest 2 scores allows us to assess whether any knowledge 
gain was lost in the time after the kit was completed. 

For each comparison, mean values were computed, and 
statistical significance was determined by using a paired 
Student's f-test for each science kit. Each comparison only 
included data from students who completed both respective 
survey instruments (pairwise valid n). Evaluating whether 
these mean differences are significant involves performing 
multiple statistical tests simultaneously, which requires a 
readjustment in the critical p value to avoid false-positive 
errors. The Bonferroni correction is the simplest, most 
conservative approach to readjust the critical p value: p = 
aln, where oc is the desired significance level (0.05), and n is 
the number of tests performed (15) (Shaffer, 1995; Weisstein, 
2011). Solving for p yields a critical p value of 0.003. Thus, for 
each comparison, p < 0.003 indicates statistical significance. 

Knowledge Acquisition (Pretest Versus Posttest 1) 

To evaluate knowledge acquisition for each kit, we 
compared student scores of the surveys given immediately 
before (pretest) and after (posttest 1) science kit instruction. 
Table III summarizes the descriptive statistics and f-test 
results for knowledge acquisition. For each kit, pairwise valid 
n ranged from 167 to 296. Mean values of pretest scores for 
each kit ranged from 0.42 to 0.60. Mean values of posttest 1 
scores for each kit ranged from 0.64 to 0.87. Each kit showed 
a positive gain in knowledge, ranging from 0.21 to 0.40. All p 
values were <0.003, which indicates that all gains are 
statistically significant at the Bonferroni-adjusted critical 
significance level. 

Knowledge Retention (Pretest Versus Posttest 2) 

A comparison of pretest and posttest 2 scores enables 
us to measure whether the newly acquired knowledge was 
retained 2 weeks after science kit instruction. Table IV 
summarizes the descriptive statistics and f-test results for 
knowledge retention. For each kit, pairwise valid n ranged 
from 166 to 294. Mean values of pretest scores ranged from 
0.43 to 0.60; these are similar but not identical to the pretest 
values shown in Table III due to slight differences in 
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TABLE IV: Knowledge retention: descriptive statistics and paired Student's f-tests of student scores on the pretest and posttest 2. 


Science Kit 

Pairwise Valid n 

Pretest 1,2 

Posttest 2 1,2 

Difference 3 

Student's f-test 

p Value 

Ocean Acidification 

166 

0.48 ± 0.20 

0.80 ± 0.22 

0.32 ± 0.22 

18.24 

<0.001 

Ocean Conveyor Belt 

294 

0.43 ± 0.22 

0.60 ± 0.21 

0.17 ± 0.23 

12.71 

<0.001 

Plankton 

206 

0.45 ± 0.17 

0.84 ± 0.17 

0.39 ± 0.22 

25.20 

<0.001 

Random Sampling 

207 

0.55 ± 0.23 

0.73 ± 0.24 

0.18 ± 0.25 

10.46 

<0.001 

Marine Debris 

257 

0.60 ± 0.23 

0.84 ± 0.19 

0.24 ± 0.23 

16.82 

<0.001 


1- The pretest and posttest 2 were given to students immediately before and 2 weeks after science kit instruction, respectively. 
2 Values are means ± SD. 

difference = posttest 2 minus pretest. Values are means ± SD. 


pairwise valid n. Mean values of posttest 2 scores for each 
kit ranged from 0.60 to 0.84. Each kit showed a positive 
knowledge gain, ranging from 0.18 to 0.39. All p values were 
<0.003, again indicating statistical significance. This analysis 
shows that even 2 weeks after kit instruction, significant 
knowledge was retained since the pretest. 

Knowledge Loss (Posttest 1 Versus Posttest 2) 

To evaluate the size and significance of any knowledge 
loss in the 2-week period after posttest 1, a second posttest 
was given, and the results of the two posttests were 
compared. Table V summarizes the descriptive statistics 
and f-test results for knowledge loss. Posttest 1 means 
ranged from 0.64 to 0.87, and posttest 2 means ranged from 
0.60 to 0.84. All kits showed slight loss of knowledge (mean 
differences of —0.01 to —0.04); however, none of these 
differences were statistically significant at the Bonferroni- 
adjusted critical value of 0.003 (All p values exceeded 0.007). 

DISCUSSION 

Assessment of Teacher Experiences 

The C-MORE science kits were borrowed by teachers at 
all grade levels, from kindergarten through undergraduate. 
Forty-five teachers who used the kits with grades 2-12, as 
well as college classes, responded to the optional kit 
evaluation. Respondents agreed that the kits were easy to 
reserve and borrow from lending libraries, that the 
instructions were easy to follow, and that kits were easy to 
use. They indicated that they would continue to use the kits 
in the future. 

The teachers successfully utilized the kits as a curriculum 
supplement in nine different subject areas, especially in 
biology, chemistry, and marine science. Presumably, teach¬ 


ers referred back to the kits as they continued their units, 
and one teacher indicated that students continued to discuss 
the kits long after the activities ended. A formal longitudinal 
survey could address whether teachers and students 
continued to refer to the kits, but this was beyond the scope 
of our evaluation. 

Assessment of Student Learning 

Survey results for all kits showed significant increases in 
knowledge from the pretest to posttest 1. Students retained 
this knowledge from the pretest to posttest 2, which was 
administered at least 2 weeks after the end of kit instruction. 
Student survey scores showed a slight, nonsignificant loss of 
knowledge between posttest 1 and posttest 2. Based on these 
results, we can conclude that using the C-MORE science kits 
resulted in the following knowledge gains on topics directly 
related to the Ocean Literacy Principles. In the Ocean 
Acidification kit, students learned how increasing CO 2 in 
the ocean can change ocean chemistry. In the Ocean 
Conveyor Belt kit, students learned about water density and 
global ocean circulation. In the Plankton kit, students learned 
about plankton and the important role they play in marine 
food webs. In the Random Sampling kit, students learned 
how a random sample could be used to study marine 
microbes. In the Marine Debris kit, students learned about 
the environmental impacts of marine debris and how the 
debris is transported. 

Overall, the mean student scores on both posttests were 
approximately 80% or greater for the Ocean Acidification, 
Plankton, and Marine Debris kits; greater than 70% for 
Random Sampling; and at least 60% for Ocean Conveyor 
Belt. Interestingly, the three kits with the highest student 
posttest scores have each undergone major revisions and 
incorporated the most teacher feedback. The Random 


TABLE V: Knowledge loss: descriptive statistics and paired Student's f-tests of student scores on the posttest 1 and posttest 2. 


Science Kit 

Pairwise Valid n 

Posttest l 1,2 

Posttest 2 1,2 

Difference 3 

Student's f-test 

p Value 

Ocean Acidification 

158 

0.81 ± 0.18 

0.79 ± 0.23 

-0.02 ± 0.21 

-1.15 

0.251 

Ocean Conveyor Belt 

270 

0.64 ± 0.20 

0.60 ± 0.20 

-0.03 ± 0.21 

-2.67 

0.008 

Plankton 

204 

0.85 ± 0.15 

0.84 ± 0.17 

-0.01 ± 0.17 

-1.05 

0.293 

Random Sampling 

202 

0.77 ± 0.23 

0.73 ± 0.24 

-0.04 ± 0.19 

-2.68 

0.008 

Marine Debris 

252 

0.87 ± 0.18 

0.84 ± 0.20 

-0.02 ± 0.16 

-2.31 

0.022 


1 Posttest 1 and posttest 2 were given to students immediately after and 2 weeks after science kit instruction, respectively. 
2 Values are means ± SD. 

difference = posttest 2 minus posttest 1. Values are means ± SD. 
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Sampling kit (posttest values of >70%) has undergone 
intermediate-level revisions, with slightly less teacher 
feedback received. The Ocean Conveyor Belt kit, which 
had the lowest posttest scores, has yet to undergo a major 
revision to incorporate teacher feedback. This appears to 
indicate that incorporating input from active classroom 
teachers results in more effective classroom tools and 
thereby increases student learning. 

One of the limitations of our study is that we did not 
formally survey teachers to discover what topics they 
covered prior to using the kits or between posttest 1 and 
posttest 2. This would have helped us determine if student 
gains were solely the result of kit use or due to a 
combination of kit use and the teacher's "normal" curric¬ 
ulum. Given the significant increase in student scores 
between the pretest and both posttests and the slight but 
nonsignificant loss in knowledge between posttests, it 
appears that the knowledge gains are the result of 
participating in the science kit exercise. 

Broader Impacts of Kit Use 

Teacher comments from unsolicited emails and informal 
interviews indicate that kit use promoted meaningful 
learning that extended beyond knowledge acquisition and 
retention evaluated in the surveys. For example, after using 
the Ocean Acidification kit, high school students contacted 
scientists and policy makers to take action to help alleviate 
ocean acidification. A teacher using the Plankton kit told us, 
"One girl was so excited about microbes and looking at 
things through a microscope that she brought in some slides 
today from home. I love it when I can get students genuinely 
excited about something—enough so that they want to take 
their own time to investigate more." Another teacher using 
this kit shared, "A couple of students and I ... Our idea is to 
study and identify the different types of plankton around the 
island of 0‘ahu." Feedback on the Ocean Conveyor Belt kit 
included "When I had [the students] list the things we've 
covered this year. Ocean Density came up without 
any prompting from me!!! Wow!! That's amazing! To 
think they've remembered that from so many months 
ago ... when they don't even remember what I talked about 
yesterday." 

The Marine Debris kit appears to have particularly 
inspired students. After engaging with the kit, students 
participated in beach clean-ups, organized clean-ups around 
their school to stop the trash from reaching the ocean, 
encouraged their families to change purchasing habits to 
reduce the use of plastic, discouraged their peers from 
littering, and educated others about the environmental 
impacts of marine debris. Thus, participating in the kits 
not only increased content knowledge, but also inspired 
students to apply their knowledge outside the classroom. 

SUMMARY 

C-MORE science kits (hands-on curriculum supple¬ 
ments on various topics in oceanography) were evaluated in 
formal classroom environments in two ways: qualitative 
assessment of teacher experience and quantitative assess¬ 
ment of student learning. The teacher surveys (n = 45) 


indicate the kits are versatile (e.g., used in a wide range of 
class types and grade levels), are easy for teachers to borrow 
and use, actively engage students, and result in meaningful 
learning (e.g., students applying the knowledge to situations 
outside of the classroom). The student evaluation (n = 
1,236), which employed a pretest, posttest 1, and posttest 2 
survey methodology, indicates significant knowledge acqui¬ 
sition (pretest-posttest 1 mean differences of 0.21-0.40), 
significant knowledge retention (pretest-posttest 2 mean 
differences of 0.17-0.39), and a slight but nonsignificant loss 
in knowledge in the 2 weeks after instruction (posttest 1- 
posttest 2 mean differences of —0.01 to —0.04). Together, the 
teacher and student evaluations indicate that the C-MORE 
science kits are effective classroom tools that can serve as a 
model for hands-on curriculum supplements. 
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APPENDIX 1 


PLANKTON SURVEY - LESSONS 1 , 2 , and 3 
VERSION 2 


Check one: 

_Pre-survey 

_Post-survey 


Name:_ 

Period: 


Directions: 

This survey is both a pre- and post- survey. Put a check mark at the top of this paper next to the survey 
you are doing (pre- or post- survey). Please answer each question to the best of your ability. Circle the 
most correct answer. 


1. What is the general term for animals that drift in the ocean? 

a. copepods 

b. holoplankton 

c. meroplankton 

d. zooplankton 

2. These organisms photosynthesize and form the base of the ocean food web. 

a. crustaceans 

b. copepods 

c. phytoplankton 

d. zooplankton 

3. Which of the following adaptations would not help phytoplankton float in the photic (light) 
zone? 

a. chlorophyll 

b. spines 

c. forming chains of cells 

d. flagella (hair-like tails) 

4. Animals in the ocean could survive without plankton. 

a. True 

b. False 

5. Which of the following shows energy flow through a simple ocean food chain? 

a. sun -> diatoms copepods fish -> whales 

b. sun -> copepods fish -^diatoms whales 

c. sun -> fish -> diatoms copepods whales 

d. diatoms sun -> fish -> whales copepods 

6. Which of the following reduces greenhouse gas in the atmosphere? 

a. meroplankton 

b. copepods 

c. zooplankton 

d. diatoms 
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7. Which best defines the difference between holoplankton and meroplankton? 

a. Meroplankton photosynthesize. Holoplankton don't. 

b. Meroplankton are smaller and more abundant than holoplankton. 

c. Meroplankton are only plankton when they are young. Holoplankton are always 
plankton. 

d. Meroplankton stay near the lit ocean surface. Holoplankton are found throughout the 
ocean. 

8. Which best describes why plankton are important to people? 

a. Plankton create much of the oxygen that humans breathe. 

b. Plankton support all the marine organisms that humans eat. 

c. Plankton are a potential source of medicine. 

d. all of the above 

9. Human activities can increase C0 2 in the ocean, and this increased C0 2 can be harmful to marine 
organisms. Which of the following may help decrease C0 2 in the ocean? 

a. phytoplankton blooms 

b. zooplankton migration 

c. meroplankton 

d. holoplankton 

10. If you were a scientist who studied phytoplankton, which tool would not help you study these 
organisms? 

a. a satellite photo 

b. a stethoscope 

c. a plankton net 

d. a microscope 

11. If copepods disappeared, which would most likely increase? 

a. fish 

b. diatoms 

c. humpback whales 

d. cnidarians 


